From: John A. Smart 1 408 490 2853 To: TC 2100 - BF 



Date: 10/17/2003 Tim*: 8:35:48 PM 



Pag* 36 of 150 



(19) 



EuropBlechee Petentamt 
European Patent Offloe 
Office ouropden dee brevets 



(12) 



(43) Dale of publication; 

12.04.2000 Bulletin 2000/1 5 

(21) Application number: 99307580.3 

(22) Dale of filing: 24.09.1999 



(ii) EP 0 992 922 A2 

EUROPEAN PATENT APPLICATION 

(51) int Cl 7; G06F 17/30 



(84) Designated Contracting States: 

AT BE CH CY DE DK ES Fl FR GB GR IE IT LI LU 
MC NL FT SE 

Designated Extension States: 
AL LT LV MK RO SI 

(30) Priority: 29.01.1999 US 236622 
02.10.1998 US 102944 P 

(71) Applicant: International Business Machines 
Corporation 

Armank, NY 10604 (US) 

(72) Inventors: 

• BhagwHl, Pravln, o/o IBM United Kingdom Ltd. 
Winchester, Hampshire S021 2JN (GB) 



• Han, Richard Y«h-wheln,c/o IBM United 
Kingdom Ltd. 

Winchester, Hampshire S021 2JN (GB) 

• La Malre, Richard O., 

a/a IBM United Kingdom Ltd. 
Winchester, Hampshire S021 2JN (GB) 
« Mummert, Todd William, 
c/o IBM United Kingdom Ltd. 
Winchester, Hampshire S021 2JN (GB) 

• Rubas, James, c/o IBM United Kingdom Ltd. 
Winchester, Hampshire S021 2JN (GB) 

(74) Representative: Davlee, Simon Robert 
IBM, 

United Kingdom Limited, 
Intellectual Property Law, 
Hursley Park 

Winchester, Hampshire S021 2JN (GB) 



(54) Automatic Image data quality adjustment to reduce response time of a Web server 



(57) The present invention provides methods, de- 
vices and systems for dynamically adjusting transcod- 
ing parameters so as to Increase the benefits of trans- 
coding. Methods of adaptation are designed to cope 
with the variability of network characteristics and of the 
size of tranecoded images. The invention also provides 
a method and apparatus to enable the transcoding 
proxy to adjust a quality-size tradeoff on a per-image 



and/or a per-cllent basis. The adaptive transcoder 
chooses different parameters tor each object, and pro- 
vides performance improvements. The invention further 
provides a general framework tor making policy deci- 
sions taking into account available bandwidth, content 
and type of image, and user preferences. The invention 
also Includes methods for generating feedback about 
the choice of optimal transcoding parameters to the us- 
er 
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Description 

[0001] This invention relates to browser proxies that transform image objects embedded inside documents. It is more 
generally related to any proxy or gateway system thai processes MIME encapsulated objects and the world wide web. 
$ [0002] Transcoding proxies sometimes act H3 intermediaries between web servers and a variety of client devices 
that are connected over communication links with widely varying characteristics. Generally, a transcoding proxy pro- 
vides two major benefits: dramalic reduction in download limes for low-bandwidth links, and tailoring of web data to 
the client device. 

[0003] Mobile devices are frequently connected via low-bandwidth wireless or medium bandwidth wireline modem 

10 links that make the viewing of rich web content, such as images, very cumbersome due to the very bng download 
times that result. In addition, the cost of such downloads can be prohibitive over tariffed wide-area wireless networks. 
Transcoding web proxies can reduce the size of web data while maintaining most of Its semantic value. Down toad time 
reductions of six to ten times may be achieved for many typical web images without losing their intelligibility. 
[0004] The second Important benefit Is that such an Intermediate proxy Is sometimes also capable of tailoring text 

1* and images for the multitude of small, weakly connected, but web-enabled, mobile devices that are now available. The 
capabilities of these mobile devices to receive, process, store and display web contenl varies widely. Given the variety 
of client devices, It is difficult for Internet content publishers to tailor content for the individual devices. Consequently, 
an active web proxy is used to transcode/change the web content to best fit the resolution, color-depth, and size 
constraints of a small-screened device and to reduce the size of the stored data to a fraction of its original size. 

20 [0005] A typical transcoding web proxy may be Implemented by adding a transcoding module in an HTTP proxy. The 
transcoding module Is an Input/output system that takes a data object (e.g., an Image or HTML page) as Input and 
transforms it into a low resolution object (e.g., a low-quality image or summarized HTML page) according to a set of 
chosen parameters. These parameters are input to the transccder which determines the resolution loss and size 
reduction of the object. The following discussion Is mostly concerned with Image objects. 

25 [0006] To maximize the benefits of transcoding, it is important to choose the quality Vs. size tradeoff euch that the 
best quality Image Is sent within the delay tolerance specified by the user. Choosing the right set of transcoding pa- 
rameters, however, is difficult. First, it is hard to estimate the size of the transcoded image size because the degree of 
compreeeion achieved through transcoding is content dependent For example, eoms JPEG files can be compressed 
by up to 90% while others may only yield Iea3 than 5% compression. The same is true when GIF^to-GIF or GIF-to- 

30 JPEG transformations are applied. Since transcoding is a compute-intensive process, processing is only worthwhile 
when high compression ratios can be achieved. For images that are already well-compressed (or of low quality) trans- 
coding may be wasteful. 

[0007] Even If the size of the transcoded output can be predicted, network variability makes It hard to predict Image 
download times. Since the user perceived quality and latency are the final performance measures, accurate estimates 
35 of download times are needed in order to choose the optimal point in the quality Vs. size tradeoff. Over best effort 
service networks, such as the Internet, accurate estimates of network characteristics (bandwidth, delay, loss rate) are 
hard to compute. Often the variance is so large that the statistical estimates are not meaninglul for making any quality 
adjustments. 

[0008] Another issue that makes policy decisions a complicated task is the location of the bottleneck in the network. 
40 when the path between the web server and the proxy is the bottleneck, transcoding does not help at all. On the contrary, 
it makes things worse due to the 3tore and forward nature of the operation. Ideally, web proxies should stream images 
from web servers to clients whenever the server to proxy path is the bottleneck. Even in cases where the proxy to client 
path is the bottleneck, any benefit due to size reduction must be weighed against the store and forward delay incurred 
due to transcoding. 

& [0009] Due to the difficulty of estimating the network characteristics and Image content, transcoding proxies may 
often only support adaptation at a very coarse level. Adaptation involves changing transcoding parameters which are 
usually selected from a predefined set of defaults, or chosen by the user. The problem with both approaches is that 
the full potential of transcoding Is not utilized. Predefined static policies are unresponsive to changes In network band- 
width and user selectable policy decisions are likely to be sub-optimal due to the lack of knowledge about the state of 

so the network. No method exists for providing feedback to the user about the combined state of the network and the 
proxy. Using iterative refinement, users can converge to the optimal choice ol transcoding parameters, but such a 
process is time consuming and not very ueeMriendly. 

[0010] Figure 1 shows an example transcoding proxy scenario 100 with varied links and varied client devices. As 
shown in Figure 1, we consider an example scenario in which all of the web requests and responses for clients 130-134 
& are passed through a generic (Hypertext Transfer Protocol) HTTP proxy 190. In the example scenario of Figure 1, it 
is the downloading of large data objects, primarily images, over the last link 160-164 from the proxy to the client that 
typically Is the main source of delay In the end-to-end response time experienced by the clients 130-134. 
[0011] The Iranscoding proxy 1 90 provides massive data thinning of wab data to enable real-time browsing of web 
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data over low-speed wide-area wireless links, like Cellular Digital Packet Data (CDPD) 163 that provides 10 kb/s or 
less ol throughput, depending upon the number of clients sharing the link. The end client could be a tull-tunction PC 
1 34 with a good color display, so that the primary problem is bandwidth reduction. Alternatively the client might be a 
small, web-enabled, mobile devices 190-132 in which case it is advantageous to tailor the web data for the specific 
* client device, particularly the client '3 display characteristics. Different aspects of transcoding proxy design benefit dif- 
ferent scenarios (i.e., tariffed proxy-client link, strong/weak client display, etc.), but all are handled well within the same 
proxy architecture 100. 

[0012] Figure 2 shows a block diagram of an embodiment ol transcoding proxy 190 used to transform objects based 
on user specified preferences and static policies. A transcoding proxy 1 90 Is built by combining a transcoding module 

io 240 with an HTTP proxy engine 220. An HTTP request 222 originates from a client 230 and is forwarded 224 by the 
proxy 190 to a web server 210. The response data 223 (i.e., HTML pages and GIF and JPEG images) are transformed 
by the tranacoder 240 and then forwarded 2SS to the client 230. Typically, a number of transcoding parameters are 
specified to the transcoder 240 in order to achieve the desired quality/size reduction of the object contained in the 
response data 226. Transcoding proxies In use today either use a static set of policies 250 or use some form of user 

1* specified preferences 260 via path 265 to determine the transcoding parameters. When a fixed set of transcoding 
parameters are applied to all objects, results are not always beneficial. In fact, in many cases, transcoding leads to 
poorer performance. 

[0010] Accordingly, in a first aspect, the present invention provides a method for a transcoding proxy to facilitate 
browsing between a plurality of client devices and a plurality of servers connected via a communication network, the 

so method comprising; receiving a request from one of the client devices tor an object stored at one of the servers, lor* 
warding the requesttortheobject to said oneof the servers, receiving the object from said one otthe servers, examining 
preferences specified by a user of said one of the client devices, examining contents of the object, examining commu- 
nication network characteristics, choosing a set of transcoding parameters, forming a transccded form of the object, 
and sending the transcoded form to said one ol the clients. 

25 [0014] Preferably, the network characteristics include bandwidth, and examining network characteristics includes 
estimating network bandwidth between said one of the servers and the proxy as well as between the proxy and said 
one of the clients. 

[0015] Preferably, the network characteristics include delay, and the step of examining network characteristics in- 
cludes estimating delay between said one of the servers and the proxy as well as delay between the proxy and said 
w one of the clients, 

[0016] The method preferably further comprises providing feedback to the user about a level of transcoding per- 
formed on the object to form the transcoded form. 

[0017] Preferably, the step ol examining Includes detsrmlnlng the size of the object. 

[0016] In the mgthod as described the object may be of type image forming an image object, and the method further 
comprises; determining dimensions of the image object, and calculating the compression ratio of the image object. 
Further to bs preferred Is that the dimensions of the Image object are determined by area of the Image In square pixels, 
and the compression ratio is determined by the bpp ratio of the image object. 
[0019] The step of forming a transcoded form preferably employs dynamic adaptation. 

[0020] The step of forming a transcoded form is preferably started before the step of receiving the object from said 
4& one of the eervere ie complete. The received object type is preferably of type JPEG forming a JPEG object. Further to 
be preferred is that the stBp of forming a transcoded form includes performing JPEG-to-JPEG image transcoding and 
the step of sending the transcoded lorm starts writing out at least one MCU of JPEG-encoded output image data before 
the step ol receiving the object is complete. 

[0021] In the method as described, the step ol sending out the transcoded form is preferably started after processing 
« an Initial fraction of the received object, and before the step of receiving the Image object from said one of the servers 
is complete. 

[0022] The step of sending out the transcoded form is preferably started before the step of forming a transcoded 
form of the object Is complete. 

[0023] In a second aspect, the present invention provides a method for a proxy to form a transcoded form of an 
so object received from a server In satisfaction of a request from a client for an object available from the server, the method 
comprising dynamically adapting parameters for transcoding the object for the client, forming a transcoded form of the 
object, and sending the transcoded form to the client. The stsp of adapting parameters preferably includes determining 
at least one characteristic of the object. One characteristic is preferably an object-header, the object- header providing 
information about the size and the type of the object. The method preferably further comprises comparing the 6ize of 
ss the object to a threshold parameter called "sizeJhresholdV The step of adapting parameters preferably includes gath- 
ering present network characteristics between the server and the proxy and between the proxy and the client. One of 
the characteristics Is preferably network bandwidth and the step of adapting Includes estimating network bandwidth 
bstwaen the server and the proxy as well as between the proxy and thB client. 
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[0024] In tie method as described, the transcoded form is preferably dependent upon the estimated bandwidth, The 
step of adapting also preferably Includes retrieving preferences ol the user, and wherein the transcoded form Is de- 
pendent upon the preferences. The step of adapting preferably includes examining the contents of the object. The 
object is preferably of type image forming an image object, and examining the content of the image object includee 

5 determining dimensions of the image. 

[0025] In the method as described, the step of adapting Is preferably dependent upon determining the compression 
ratio of the image object. The type of image object is further preferably GIF and wherein the step of adapting is de- 
pendent upon comparing the compression ratio against a predetermined policy threshold called 'giUhreshold", 
[0026] The method as described preferably further comprises predicting at least one parameter of the transcoded 

io form of the object. The transcoded form is further preferably the same as an original form of the object. Preferably, at 
least one of the parameters includes a size of the transcoded lorm. Preferably also, at least one of the parameter© 
Includes the time spent In transcoding the object. 

[0027] In a third aspect, the present invention provides a method for predicting parameters of a transcoded form of 
an object, the object having an Initial size and dimension, and the object being received Irom a server In satisfaction 
75 of a request Irom a client for the object, the method comprising: computing the bpp ratio of the object, gathering a set 
of statistics ol a plurality of previously tranecoded objects, and employing the set of statistics and the bpp ratio for 
predicting the parameters. 

[0028] In a method as described, at least one of the parameters is preferably size and the set of statistics includes 
sizes of a plurality of previously transcoded objects statistics. 
so [0029] The object is preferably of type image and the set of statistics includes image quality. The pi uralityol previously 
transcoded objects are preferably chosen from a predetermined benchmark suite of Images. 

[0030] In a method as described, the step of employing preferably uses dynamically updating the set of statistics 
using the statistics of the currently transcoded object. At least one of the parameters is preferably a duration for forming 
a transcoded form of an object and the set of statistics Includes the duration for forming a plurality of previously trans- 
2S coded objects. 

[0031] In afourth aspect, the present Invention provides a transcoding proxy system for facilitating browsing between 
a plurality of clients and a plurality of servers connected via a communication network, the proxy comprising an HTTP 
proxy engine to receive a request from one of the clients for an object stored at one of the servers, and to fetch the 
object from said one of the servera, an object transcoder to form a transcoded form of the object using a set of param- 

w eters for transcoding, a dynamic policy module to determine the set of parameters of transcoding, an image size and 
delay predictor module to gather characteristics of the object, a user preference module to gather quality preferences 
specified by a user of said one of the clients, and a bandwidth estimation module to estimate available network band- 
width, wherein dynamic policy module dynamically adjusting the parameters of transcoding using the Input received 
from the image size and delay predictor module, user preferences module, and bandwidth estimation module for the 

35 purpose of improving satisfaction for the user, and the transcoding system providing feedback to the user about the 
level of transcoding performed. 

[0032] The user preferences module preferably further collects the characteristics such as display size, resolution, 
& CPU speed of said one of the devices, and provides those characteristics to the dynamic policy module. 
[0033] In the system as described, the bandwidth estimation module preferably collects traces of previously estab- 
40 lished network connections between said one of the servers & the proxy, collects traces of previously established 
network connections between the proxy and said one of the clients, and estimates the object download time by per- 
forming statistical analysis on the collected traces. 

[0034] The statistical analysis used for estimating bandwidth between said one of the servers and the proxy is pref- 
erably based on computing a statistical measure such as median : mean, or mode of download times of previously 

'5 fetched objects as determined from the collected traces. 

[0036] The statistical analysis used for estimating bandwidth between the proxy and said one of tha clients is pref- 
erably based on computing aggregate bandwidth of all active connections between the proxy and said one of the 
clients. The system preferably further comprises displaying a slide bar on said one of the client's display for collecting 
the user specified preferences. In a system as described, the user of said one of the clients can preferably specify the 

so tradeoff between download time and data quality through the use of a graphical user Interface with a slide bar. The 
user of said one of the clients can preferably specify through the use of a graphical user interface with a slide bar the 
tradeoff between download time and image quality including a specific switch to select color or gray scale as the desired 
output form. The user of said one of the clients can further preferably specify through the use of a graphical user 
interface with a slide bar, the desire to maintain a target response time such that the system automatically reduces 

S& data quality (and hence data download size) to compensate for dynamic variations in bandwidth to said one of the 
clients. The graphical user interface slider bar is also preferably used as an output interface for showing the optimal 
choice of transcoding parameters to the user. 

[0036] In a fifth aspect, the present invention provides an article of manufacture comprising a computer usable me- 
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dium having computer readable program code means embodied therein for causing dynamic adaptation of transcoded 
form of an object In a transcoding proxy, the computer readable program code means In aald article of manufacture 
comprising computer readable program code moans for causing a computer to effect: a proxy receiving an object 
associated with a user from a server, determining parameters of the object, retrieving preferences of the user, gathering 
* present network characteristics, obtaining transcoding policy thresholds, performing a policy decision based upon ob- 
ject parameters, user preferences, network characteristics, and policy thresholds, forming atranscoded object, pro- 
viding feedback ol a level of transcoding performed on the object to the user, and sending the transcoded object to the 
user. 

[0037] In a sixth aspect, the present Invention provides an article of manufacture comprising a computer usable 
to medium having computer readable program code means embodied therein for causing a transcoding proxy to facilitate 
browsing between a plurality of client devices and a plurality of servers connected via a communication network: the 
computer readable program code means In said article ol manufacture comprising computer readable program code 
means for causing a computer to effect: receiving a request from one of the client devices for an object stored at one 
of the servers, forwarding the request for the object to said one of the servers, receiving the object from said one of 
75 the servers, examining preferences specified by a user ol said one of the client devices, examining contents of the 
object, examining communication network characteristics, chooeing a set of transcoding parameters, forming a trans- 
coded form of the object, and sending the transcoded form to said one of the clients. 

[0036] The computer readable program code means in said article of manufacture preferably lurther comprises com- 
puter readable program code means for causing a computer to effect providing feedback to the user about a level of 

20 transcoding performed on the object to form the transcoded form. The computer readable program code means in eaid 
article of manufacture preferably further comprises computer readable program code means for causing a computer 
to effect determining dimensions of the object, and calculating the compression ratio of the object. The computer 
readable program code means in eaid article of manufacture preferably further comprises computer readable program 
code means for causing a computer to affect starting the step of forming a transcoded form before the step of receiving 

25 the object from said one of the servers is complete. 

[0059] In a seventh aspect, the present Invention provides a computer program product comprising a computer 
usable medium having computer readable program code means embodied therein for causing a proxy to form a trans- 
coded lorm of an object received from a server in satisfaction of a request from a client for an object available from 
the server, the computer readable program code means in 3aid computer program product comprising computer nead- 

30 able program code means for causing a computer to effect: dynamically adapting parameters for transcoding the object 
for the client, fanning a transcoded form of the object, and sending the transcoded form to the client. 
[0040] The computer readable program code means in said computer program product preferably further comprises 
computer readable program code means for causing a computer to effect gathering present network characteristics 
between the server and the proxy and between the proxy and the client. The computer readable program code means 

?s in said computer program product preferably further comprises computer readable program code means for causing 
a computer to effect adapting parameters for transcoding based upon the estimated bandwidth and preferences of the 
user. 

[0041] In an eighth aspect, the present Invention provides a program storage device readable by machine, tangibly 
embodying a program of instructions executable by the machine to perform method steps for predicting parameters 
*o of a transcoded form of an object, the object having an initial size and dimension, and the object being received from 
a server in satisfaction of a request from a client for the object, said method steps comprising: computing the bpp ratio 
of the object, gathering a set of statistics of a plurality of previously transcoded objects, employing the set of statistics 
and the bpp ratio for predicting the parameters. 

[0042] Said method step ol employing the set of statistics preferably further comprises updating the set ol statistics 
using the statistics of the currently transcoded objects. 

[0043] An aspect of the present invention is to provide methods, devices and systems for dynamically adjusting 
transcoding parameters so as to increase the benefits of transcoding. Methods of adaptation are designed to cope 
with the variability of network characteristics and of the size of transcoded Images. 

[0044] In an embodiment, the invention includes three new components: an image size predictor, a network band- 
60 width (bAw) analyzer, and a policy module. Before Initiating any transcoding action, the policy module queries the Image 
size predictor to estimate the size of the output image. The b/w analyzer is queried to collect an estimate of the image 
transmission time from the server to the proxy, and from the proxy to the client. Based on the collected estimates, the 
proxy decides whether or not to tranacode the Image. In addition, the policy module can also compute the optimal point 
in the quality vs. size tradeoff that would provide the user specified performance criteria (e.g., reduced response time, 
increased quality). 

[0045] Another aspect of the present invention is to provide a method and/or apparatus to enable the transcoding 
proxy to adjust a quality-size tradeoff on a per-tmage and/or a per-cllent basis. The adaptive transcoder chooses dif- 
ferent parameters for each object, and provides performance improvements. 



5 

Received from < 1 408 490 2853 > at 10/18/03 12:50:19 AM [Eastern Daylight Time] 



From: John A. Smart 1 408 490 2853 To: TC 2100 - BF 



Data: 10/17/2003 Time: 8:35:48 PM 



Paga 41 of 150 



EP 0 992 922 A2 

[0046] Still another aspect of this invention provides a general framework for making policy decisions taking into 
account available bandwidth, content and type of Image, and user preferences. The administrator of the proxy chooses 
from a variety of optimization objectives so as 1o obtain improved performance from the system. In one embodiment 
when the proxy tranacoder throughput is the bottleneck, the policy module ie instructed to judiciously use CPU reaourcee 
* so aa to reduce the response time for all users. An advantageous element of the invention is the automated nature of 
decision making, to free up users from actively controlling the policy engine of the proxy. 

[0047] In still another aspect of the present invention a method is provided for generating feedback about the choice 
of optimal transcoding parameters to the use r. In an example of embodiment, the transcoding system provides feedback 
to the user by dynamically adjust I ng the position of the user preference allder bar. The user preference slider bar serves 
10 both as an input as well as an output device. 

[0048] A prelerred embodiment of the present invention will now be described by way of example: with reference to 
the drawings In which: 

• FIG. 1 shows an example transcoding proxy scenario with varied links and varied client devices; 

FIG. 2 shows a block diagram of a transcoding proxy used to transform objects based on user specified preferences 
and static policies; 

FIG. 3 shows a bbck diagram of an example of a transcoding proxy modified in accordance with the present 
20 invention to Include an image size predictor, bandwidth estimator, dynamic policy module, and a user feedback 

generator: 

FIG, 4 shows an example block diagram of an example HTTP proxy with caching and transcoding modules in 
accordance with the present Invention; 

25 

FIG. s shows an example flow diagram of a transcoding dynamic policy module In accordance with the present 
invention; 

FIG. 6 3hows an example web request-response cycle using a transcoding proxy in accordance with 1he present 
&o invention; 

FIG. 7 shows an example regime where transcoding is useful in accordance with the present invention; 

FIG. 9 shows an example flow diagram of a example policy function in accordance with thg present invention; 

35 

FIG. 9 shows an example block diagram of an example Image size prediction module In accordance with the 
present invention; 

FIG. 10 shows an example block diagram of an example bandwidth prediction module in accordance with the 
4 & present invention; 

FIG. 11 shows inputtfeedback user interface in accordance with the present invention; and 

FIG. 12 illustrates an example timing diagram of streaming transcoding in accordance with the present invention. 

45 

[0043] Figure 3 shows a block diagram 300 of an example embodiment of changes to transcoding proxy 1 90 in 
accordance with the present invention. A comparison of Figure 3 to Figure 2 shows a replacement of the static policy 
module, 250 In Figure 2, with a dynamic policy module, 370 In Figure 3. A purpose of the dynamic policy module 370 
ie to make decisions concerning when to turn transcoding on and off and what transcoding policy (i.e., the transcoding 

so algorithm along wfth Its parameters) to use. The dynamic policy module 370 also Interfaces with an Image size and 
delay predictor 375, a bandwidth estimator 360, and a user feedback provider 390. In the embodiment shown, the 
policy module 370 employe a number of criteria, including: the characteristics of the data (e.g., size of the images, 
current encoding efficiency, structural role in the HTML page) as determined by the content analysis flow diagram 
(shown in Figure 5) : the current estimate of the bandwidth on the proxy-to-client and server-to-proxy links (shown in 

ss Figure 10), the characteristics of the client, particularly the client display capabilities, and the user preferences con- 
cerning the preferred rendering of the data (shown as the user slide bar preferences in Figure 11). 
[0060] The Items shown In Figures 5, 10 and 11 are described below. In particular, the user slide bar preferences of 
Figure 1 1 , provide a method of interacting with the transcoding proxy so as to dynamically change the tradeoff betwean 
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image quality and download time, In addition to serving as an input interface, the slider bar (1140 1150 1160 of Figure 
1 1 ) also acta as an output Interface, displaying the feedback 390 that Is received from the dynamic policy module 370. 
[0051] Figure 4 shows a block diagram 400 illustrating an example function of a multi-resolution cache 410 in a 
tranacoding HTTP proxy eystsm 400 in accordance with the preaent invention. Caches are uaeful in HTTP proxies to 

$ provide reduced response time for repeated data requests (by the same or different clients) for the same data object. 
A variety of methods may be used to assure that the cached data is up-to-date. In the example caching transcoding 
system 400, the multi-resolution cache 410 stores an original version of the data, as well as other possible forms of 
the data : including reduced resolution versions that have been transcoded for specific device types. 
[0052] As an example, we again consider the case of an Image data object, but It Is understood by those skilled In 

to the art that these methods can be applied to other data types. We describe a method for storing, tagging and retrieving 
various forms of the data in the context of a caching transcoding proxy system. 

[0063] Referring to Figure 4. It la assumed that a JPEG Image Is received from the server 210 In response to a 
request by a client 230. This image is decoded from the lossy JPEG encoding format to a bit-map representation of 
the Image. Those skilled In the art ol Image processing recognize that the JPEG encoding standard describes Images 

1* using coefficients that weight Discrete Cosine Transforms (DCTs) so a compute-intensive decoding step is generally 
required to obtain the actual colors of each pixel. Thia bit-map requires a larger storage size than the original JPEG 
image. However, given an adequate size cache it may be worthwhile to store (via data path 420) this expanded size 
image so as to avoid the compute-intensive step of JPEG decoding when a transcoded version of the same image is 
later required but with different transcoding parameters than ths first request. In a second step, based on the transcoding 

20 parameters (i.e., color depth, scaling factor and the JPEG quality factor) the JPEG image is re-encoded. The image 
may be re-encoded as a JPEG Image or In an alternative encoding format The final transcoded version of the Image 
is also stored in the multi-resolution cache 410 via data path 420. 

[0054] In the example embodiment, when additional data requests occur, the HTTP proxy 220 first checks its cache 
410 to see If an "up-to-date" version of the data object Is available at the requested resolution or transcoding level. 
25 Each object in the multi-resolution cache 410 is stored with a version specifier that includes: a URL description, a time 
stamp for the data object, and the object characteristics. For a JPEG Image the object characteristics Include color- 
depth, seating factor, and JPEG quality factor. An alternative embodiment has an indication that thB JPEG image has 
been decoded and stored in its bit-map form, or that it has been converted and stored as a GIF with various charac- 
teristics. 

so [0055] If an up-to-date version of the object is available with the requested type and resolution, then that version is 
returned to the client. If this is not available, but an up-to-date version of the object exists in either its original JPEG 
form or the decoded bit-map form, then this version is returned to the transcoder 240 with an indication of its charac- 
teristics. This enables the transcoder 240 to produce the desired version of the object, which la rsturned to the client 
230 and stored in the multi-resolution cache 410 via data path 420. 

3$ [0056] It is noted that there are several extensions to this scheme that are obvious to those skilled in the art. One 
extensions uses an already transcoded version of the object, rather than to the original data object, to generate a 
further resolution-reduced version of the object. Methods for managing the different resolution versions of data objects 
In a cache are further described In R. O. LaMalre and J. T. Robinson, "Conserving Storage Space by Means of Low 
Resolution Objects", docket YQ997308, US patent application filed February 13, 1998. 

*o [00S7] Figure S shows an example flow diagram for the dynamic policy module 370 of Figures 3 and 4. Figure 5 also 
shows how the policy module 370 interfaces with the HTTP proxy engine 220 and thB object transcoder 240 of Figure 
3. The methods described below apply to many content types including text, images : audio : and video. However, the 
following discussion locuses on image data types only. It is evident to those skilled in the art that concepts and dynamic 
policies are applicable to other media types. 

« [0058] Figure 5 shows that based on the response header received from the server 21 0, the HTTP proxy engine 220 
first determines the size of the object 510. If ths content -type of tha response is "image/* 1 520, the proxy engine 220 
passes the handle for the object to the dynamic policy module 370 for further analysis. Inside the policy module 370, 
the size of the Input object Is compared against a pre configured threshold called "alzajhreahold" 530. If the objects 
is smaller than "6ize_threshold\ or ihe content type is not "image/*" 520, then the object is no: transcoded, but instead 

w la forwarded 51 5 to the client without any content modification. Small objects (such as bullets, thumbnails, logos, etc.) 
found on the web are typically GIF objects which are already well compressed due to GIF encoding. Transcoding such 
objects does noi generally yield further compression. 

[0059] Wb convert GIF Images to GIF or JPEG Images that are reduced in size and/or color-depth (the choice of 
GIF or JPEG as the end format depends on the image characteristics). In addition, we convert JPEG images to JPEG 
SS images that are reduced in JPEG quality, size, and/or color-depth. JPEG quality refers to a transcoding parameter that 
is used to determine the degree to which the coefficients of the Discrete Cosine Transformations used in the JPEG 
encoding standard are quantized. It has been found that the JPEG quality parameter Is aleo a good predictor of per- 
ceived image quality. This parameter varies in the range 1 to 100, where 1 00 represents very high quality. JPEG images 
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found on the web typically have a JPEG quality parameter ot 75 

[0060] For large Images, the type ot the Image coding (JPEG or GIF) and the efficiency of coding are Important 
factors in transcoding decision making. Since JPEG is a lossy compression method, size reduction is always possible 
by reducing quality factor. Similar quality reduction, however, cannot be applied to GIF filea aince GIF is a lossless 
* compression method. To achieve quality reduction, a GIF file mu3t first be decoded and then re -encoded as a quality- 
reduced JPEG image, This method, however, may not always provide size reduction, GIF format is usually more efficient 
for coding maps, logos, and drawings while JPFG is more efficient for coding natural images. Converting GIF to JPEG 
is uselul only when the original GIF image is not efficiently coded. 

[0061] we define bite per pixel (bpp) as a measure of the compression efficiency. Bpp le computed ae the ratio of the 
70 image file size to the image area in pixels. In processing step 540, X and Y dimensions and the bpp value of the input 
image are computed by parsing the image header. If the input object is of type "image/jpg" 550, transcoding ie always 
performed. -However, If the content type Is D lmage/glf n 525, only those objects which yield a bpp ratio larger than 
■gif .threshold 1 535 are transcoded. GIF files that are not very efficiently encoded yield a bpp value that is larger than 
"git ..threshold". Thus, the decision step 535 Is very effective In Identifying compressible GIF flies with high accuracy. 
Though not show in Figure 5, it is evident to those skilled in the art that other transcoding policies, such as scaling and 
file truncation (lor progressively encoded data) can be uead for well -compressed GIFa. 

[0062] An important aspect of the proposed Invention is that decision steps 510-565 are carried out as soon as the 
image header is received. If the decision is to not transcode, image segments can be forwarded as soon as they are 
received from the server without incurring store and forward delay. Similarly, when transcoding is to be performed, 
x> images can either be buffered and then transcoded (store and forward transcoding), or each segment can be transcoded 
on-the-fly (streaming transcoding method). 

[0063] After identifying an image that is compressible, the next step involves detgrmining the extent to which the 
selected image should be transcoded, The policy function 585 is responsible for collecting input from three different 
sources (Image size predictor 375, bandwidth estimator 380, user preference selector 260) and subsequently selecting 

25 transcoding parameters in accordance with the steps shown in Figure 8. The chosen parameters determine the extent 
and types of compression performed by the object transcoder 240. For example, the scaling parameter determines 
how much an image is downsampled. Quantization parameters control how an image is quantized in the pixel domain 
and/or the frequency domain. The number of colors in a color mapped image can be reduced, or a 24-bit color image 
may be converted to 6-bit grayscale, or even a monochrome representation. The process of transcoding is performed 

30 in step 570 and the output of the transcoder is forwarded to the client 230. 

[0064] An important aspect of the policy function 565 is the analytical framework for making transcoding decisions. 
The analytical framework takes into consideration factors such as available bandwidth, type and size of the image, 
user preferences and provides an objective criteria for making transcoding decisions. As an example, we consider the 
object ivg of minimizing response time for the user, but it is understood by those skilled in the art that using the same 

53 framework other optimization criteria can also be applied. We describe a method for determining when it is beneficial 
to transcode, and to what extent transcoding should be applied. The embodiment described herein Is referred to as 
dynamic adaptation of transcoding parameters. 

[0065] Figure 6 shows an example web request-response cycle and the response time of fetching an object of size 
S through a store-and-forward transcoding proxy 190. We define a store-and-forward image transcoder as an image 

*o transcoder which must wait to accumulate an entire input image before transcoding can begin on thie image and then 
must wait to generate a transcoded imaga in its entirety before it is made available to be output. As shown in Figure 
6, the original image 520 of size S (bytes) 570 is downloaded into the store-and-forward proxy over the server-proxy 
connection with effective bandwidth (bits/sec). The transcoder introduces a delay DJS) 650 and generates an 
output image 630 ot size $ X (S) 630, Both the transcoding delay 650 and output image's byte size 690 are denoted to 

45 be dependent upon the Input Image's byte size S 670. The transcoded Image Is then transmitted over a proxy-client 
connection having effective bandwidth 

[0066] The policy function needs to weigh the cost (delay) of transcoding against any size reduction achieved by 
transcoding. For transcoding to provide benefits, delay Introduced due to transcoding must be offset by the reduction 
in transmission time due to compression. For very low bandwidth proxy-client access links, the reduction in response 
so time due to aggressive Image compression typically far outweighs the addition to response time caused by compute- 
intensive transcoding. However, Figure 7 shows that as the bandwidth of the proxy-client link increases, there comes 
a point (transcoding threshold 71 0) at which it ie no longer beneficial to transcode since the reduction in response time 
due to aggressive compression decreases as a function of the bottleneck link's bandwidth, while the transcoding time 
remains constant. 

5& [0067] Suppose R e is the response time of fetching a web object of size S from the web server with transcoding 
turned oft. Similarly, let denote the response time of fetching the transcoded version of the same web object through 
the transcoding proxy. For the purpose ot the following discussion we assume that caching Is not supported at the proxy 
[DOSS] The client perceived response time with transcoding turned off is the sum of the following three terms: 
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R Q = 2 + RTT pg +2* RTT^ + S/minfB^ B^ 

[0069] RTTpc is the network roundtrip time latency between the client and the proxy and, similarly, RTT ep ia the 
5 latency between the proxy and the server Fetching the web object requires a TCP SYN/ACK exchange as well as an 
H7TP request/response, thereby contributing 2* RTT^-t 2* RTT^io the delay term. In addition, a web image incurs 
a transmission delay equal to the spread In time between arrival ot ltd first and the last bite. Let mln (B^ B^J denote 
the bottleneck bandwidth between the client and the server In the absence of a proxy, the first and the last bits of an 
image will be spread in time by S/min (B^ B^. This spread corresponds to the effective transmission time ot the 
to image over the concatenated sarver-to-proxy-to-cliant connection. 

[0070] when transcoding is turned on : the proxy operates in a store and forward mode. 2 * RTTpc +2* RTTgp is 
again the fixed component of the response time. D p (S) le the additional term that represents the transcoding delay. 
The resulting response time for the transcoded object can be expressed as; 

15 * p = * * HTT^ 4- 2 * RTT W + D ff (S) + SpfSyB^ 

Transcoding will reduce response time ir R p < R 0 . That is, 

20 d p (S) ^sm^t s^syB^ < smiw^ b^) 

Clearly, when B po > B 9pi R p is always greater than R^ On the other hand, when B p9 < B^ transcoding is useful if and 
only if: 

25 

[0071] The above equation precieely characterizee the regime in which traneooding reducee reeponee time. Figure 
30 9 shows a flow diagram of an example policy function constructed using the analytical framework described above. 
810 marks the entry point of the policy function. The policy function 800 is called from 555 with the original image object 
size aa one of the Inpute to the policy function. Variable S le eel equal to the Input object elze and the quality factor q 
is initialized to the best possible initial image quality (for example, the quality of the input image). In step 630, the policy 
function BOO issues a query to the bandwidth estimator 380 asking for the estimated download time ot the object size 
s from the specified server to the proxy, referred to as T^S). It also asks tor the estimate of the download time for the 
same object from the proxy to the client, referred to as T 2 (S), Based on the logs of previous connections to the chosen 
destination, the bandwidth estimator returns an estimate of TrfS) and TgS). In the next step 640, the policy function 
queries the image size & delay predictor 375 to find an estimate of the transcoded image size S p (S). It then computes 
the estimated download time savings by subtracting the download time estimate of the transcoded image size T £ (S p 
40 ($)) from T E (S) 850. Finally, in step 970 the two quantities (transcoding delay + TrfS) - the download savings) and the 
target response time reduction are compared. If the first term is leas than the second term, computation is stopped 
and the chosen quality factor q is returned aa an output fiSO. Otherwise, the loop 840-370 is reentered with a reduced 
g value. 

[0072] It Is noted that more efficient search techniques or variants of objective functions can be designed by those 
45 skilled in the art without departing from the spirit of the policy function framework presented in this invention. One 
extension would be to rearrange the terms in the policy equation as follows: 

Response time reduction(q) = {S - S p (S))/B po - D p (S) - 

so 

[0073] In the above equation, S p (S) is also a function of the quality factor (the smaller the output size, the poorer the 
quality). Note that the transcoding delay has been found to be effectively independent of the quality factor There are 
two independent variables in the above equation; q, the quality factor, and the target response time reduction. Several 
different policies can be developed within the framework of the above equation. For example: 

ss 

Minimize response time for all users; 

Maximize quality for a us Br specified response time constraint; 
Optimize overall system performance, not just one user. 
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[0074] Since transcoding is a compute- intensive task and the proxy only has limited CPU power, the proxy's CPU 
cycles are best spent on the set of tasks that maximize overall user satisfaction. During the peak usage period, If every 
user turns transcoding on, there may not be sufficient CPU cycles available to satisfy all transcoding requests. By 
delegating the responsibility of parameter selection to the policy fund ion 800, the system can automatically select the 

$ optimal transcoding parameters and provide consistent and predictable response to the end user. If users were to 
select these parameters independently, the transcoding system could not operate at the optimal performance level. 
[007E] Let us now consider the transcoding delay and size estimation. The decision 3tep 670 requires two additional 
inputs; an estimate of the size of the transcoded image; S p (S), and the transcoding delay, D p (S). that is incurred during 
the process of transcoding. An accurate prediction of these two quantities Is difficult to obtain since they depend on 

io the image content, type of coding used, original quality factor, processor speed, and the load on the proxy. As was the 
case for the bandwidth estimators, the present invention usee statistical methods to estimate these two quantities. 
[0076] Figure 9 Is a block diagram of an example of the Image size prediction module In accordance with the present 
invention. Ag shown in 910, we initialize our system with statistical characterizations of S x (q) iha\ are based on a large 
bench mark set ol Images. The method shown In 91 0 Is used to Initialize the transcoded Image stze distribution functions 

7* that depend on the parameters cppand q. The proxy continually adds new sample points 920 to the distribution function 
as it fetches and transcodes new image objects. The function of 920 enables the image size predictor 375 to adapt ite 
behavior to different data sets 1hat a client may be accessing. 

[0077] Thus, in the method described in 930, we use statistical methods to estimate the delay and output size and 
use the bpp ratio as the criteria for performing a more accurate classification of images and hence of forming a better 
so estimate ol the resulting transcoded file size. This method works particularly well for GIF files, 

[0078] Using similar statistical methods, D p (S) can be estimated based on benchmark Initialization and on-line up- 
dating of the statistical model, we have also determined from our statistical work, thai the transcoding delay is well 
characterized as being a linear function of the number of pixels (as opposed to file size), 

[0079] Now we consider the bandwidth and download time estimation. The accuracy of decision step 370 also de- 
& pendeto a large extent on the accuracy of image download lime estimates. Bandwidth estimator 380 shown in Figures 
3-5 Is responsible for providing these time estimates. The bandwidth estimator 360 records every packet send and 
receive event at the proxy and performs statistical analysis on the collected traces to make predictions about the state 
of the network. Those skilled in the art of network monitoring recognize that the network monitoring function can be 
implemented in several different ways. One implementation would be a shim layer between the proxy application and 
the socket layer. Another implementation would be a processing layer above any packet filtering software. Generally 
the implementation built over the packet filtering software will provide more accurate estimates of bandwidth and down- 
load time. The present invention presents no requirement that the bandwidth monitor 380 and the policy decision step 
870 should both run on the same machine, though collocating the two modules minimizes the overhead of communi- 
cation between them. 

& [0080] Figure 1 0 shows an example bandwidth estimator module 330 in accordance with the present invention. The 
module consists of two components; trace monitor 1010 and a statistical analyzer 1 020. The trace monitor continuously 
monitors all connections between the clients and the proxy. The same trace monitor (or another instance of it) is also 
used to monitor the traffic between the proxy and all web servers. For each monitored connection the following Infor- 
mation is recorded: 

40 

time when a new connection is established, 

source address, source port number and destination address, destination port number of each network connection; 
number of bytes sent and received on each connection and the respective time stamp ol each send and receive 
event, 

as time when connections are closed. 

[0081] The statistical analyzer 1020 maintains a database of all past and all current active connections. For each 
connection It also keeps a history of when and how many bytes were sent and/or received on that connection. Based 
on this history the statistical analyzer 1020 is able to make predictions about the download time of lutura connections. 

so For predicting server to proxy download time and proxy to client download time we use different heuristics. 

[0082] Server to proxy traces exhibit behavior that is common among wide-area TCP connections. It has been ob- 
served ihat sequence Ve timestamp traces are non-linear and we have found that linear estimation concepts such as 
average or median bandwidth dont yield accurate estimates. To predict the download time of an object of a gK/Bn size, 
bandwidth estimator looks at the history of connections to the chosen destination and computes the distribution function 

55 of the download time for all objects that have roughly the same size. The median, or some other appropriate statistical 
function of this distribution, is' returned as the download time estimate. 

[0083] In contrast, proxy-to-client TCP behavior Is dominated by the effects of having a bandwidth constrained link, 
which is typically the last hop 1 60-164. Because there is a bottleneck link, the aggregate of all active TCP connections 
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to a client typically saturate the bottleneck link. Thus, for providing proxy to client download time estimates, bandwidth 
estimator aggregates all active connections to a client Into a single group. For each group of connections It plots a time 
vs number of bytes pbt and then performs a linear curve-fit on the data. This technique provides accurate estimate of 
the current available bandwidth between the proxy and the client. Although this value changea with time, the oecillatione 
$ in most cases are bounded. For example, bandwidth monitor can easily detect whether the client is connected by a 
10Mb ethemet, 14.4 modem, or 28.8 modem by analyzing at the output of the curve fitting algorithm. The predicted 
values are not exact (20- 30% deviation from the correct values), but those skilled in the art recognize that accuracy 
of bandwidth estimation can be improved by refining the estimation algorithm. 

[0084] Consideration Is now given to the transcoding user Interface. Transcoding user Interface Is the Interface 

io through which the user can dynamically change the tradeoff between image quality and download time. It is represented 
as the item 250 in Figure© 2-5. Figure 11 shows an example implementation of the user preference interface in ac- 
cordance with the current Invention. This user Interface Is represented well as a linear slide bar, which can be Imple- 
mented as a Java applet, to allow users to continuously vary the image quality. As shown in Figure 11 an option for 
selecting color (vs. gray scale) Is also provided. Further, an option for selecting autopilot can be used In the following 

1$ two ways; 1 ) as an indication to the proxy that it can automatically decide whether or not to perform transcoding, based 
on the current estimates of the client-proxy and proxy-server link bandwidths and currently available CPU resources 
as discussed above, or 2) as an indication to the proxy to try and maintain the current download times that the user is 
currently experiencing (i.e., by varying image quality at the proxy to compensate for dynamic bandwidth changes that 
are detected by the bandwidth estimation subsystem), or both 1) and 2) . On occasions when the user requires a full- 

w fidelity version of the data : they can easily request the refined data by adjusting the slide bar (downwards) and request- 
ing the higher quality data. Typically, we find that almost all of our web browsing can be done by transcoding Images 
to a sixth or tenth of their original size, and only rarely do we actually find a object, such as a map, that wo need to 
request a refinement of. Note that alternative HTML-only interfaces can have been invented for supporting non-Java 
devices such as windows CE based platforms. 

2$ [0085] Now we consider streaming image transcoding. The above description is generally applicable to a store-and- 
forward transcoding proxy. Following, we describe conditions under which It Is beneficial for a streamed Image trans- 
coder to engage in transcoding. A streamed image transcoder is an image transcoder which starts writing out image 
data encoded in an output format before having fully read in the complete input etream of bytee corresponding to the 
entire image encoded in the input format. 

50 [0086] In Figure 12 we illustrate an example of the reasoning behind our algorithm by using a timing diagram. The 
input image arrives as a stream of bits spaced apart by The streaming image transcoder will take a group of G 
bits for transcoding, incurring a small store-and-forward delay D v The group ol bits are then transcoded into a group 
of G p output bits, Incurring a delay It DgcD^ then the Image transcoder can convert each Input group G to Its 
corresponding output group G p before the next input group G needs to be processed. In this case, the streaming image 

re transcoder's internal memory requirement is bounded, However, if Dg>D v then the image transcoder will not be able 
to process Input bits fast enough. In this latter case, given a continuous Input stream, then the Image transcoder's 
internal memory requirement grows without bound, i.e. the image transcoder's finite-length internal RAM buffers will 
overflow. Therefore, we desire than the transcoding delay D s satisfy D^D,. Clearly, D t = G/B^. To find let O p (S) 
= the predicted image transcoding tima for an image of S bits (D p (S) actually depends on other parameters, such as 

40 image content and dimension, but we use D p (S)tor simplicity of notation). Then, D^= D p (By(S/G). To avoid overflowing 
the RAM buffer, the group transcoding delay must satisfy D p (S)/(S/Q) < G/B^ or 

D p (S) < S/B^ (Condition A) 

45 

[0007] Assuming that Condition A holds true, then the output transcoded groups G p will be uniformly spaced by a 
delay equal to D v The transmission channel can send each transcoded group of bits G p in time D 3 = Gp/B^ In case 
I), we Illustrate D 3 (f)< O t! I.e. each output group G p can be sent before the next output group Is ready for transmission. 
In case ii), D 3 (u) > D p so that the output transmission link cannot send the produced bits fast enough to keep the output 
so queue empty. In case ii) : the transmission link's output queue grows without bound given a continuous stream of trans- 
coded bits, causing overflow for finite -length link buffers. Therefore, we desire that the delay caused by the transcoded 
output group size DJi) < D v Clearly, D 3 (i)= Gp/B^ To avoid overflowing the transmission link's output buffer, the 
transcoded output Image group size Q p must satisfy Qf/B po < G/B^ or 

ss 

o > B pf /B pc (Condition B) 
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where c = group image compression ratio G/Gp, which we assume to be on average equivalent to the overall image 
compression ratio. In summary, the streamed Image transcoder should only perform transcoding when both Condition 
A and Condition B are satisfied. If the proxy-server link is the bottleneck, i.e. B qp < B po , then Condition B reduces to c 
> N, where N is a number leae than 1. Normally, the compression ratio ie alwaye greater than 1, so Condition B will 

* always be satisfied. In this case, only Condition A must be satisfied in order for transcoding to not be disadvantageous. 
In fact, when the proxy-server link is the bottleneck, Condition B could be interpreted as providing an upper bound on 
the ratio of expansion allowed for a transcoded image, namely 1/c < BpJB^. Expansion of an image may occasionally 
be necessary when format conversion is mandatory, e.g. GIF->Pa!m format. The above equation allows us to determine 
when such lormat conversion will Increase the chances of buffer overflow, and when formal conversion will not cause 

to buffer ovarfiow. For example, if B sp = 7 b ps , B^ = 2 b fi9 and Q = 1 bit, then Condition B says that the output group G p 
can expand to a maximum of 2 bits, If the client-proxy link is the bottleneck, i.e. B ap > B^ then Condition B says that 
the Image compression ratio c must be greater than the ratio of proxy^erver to client-proxy bandwldths In order for 
transcoding to be worthwhile. In addition, Condition A must still be satisfied. 

[0088] It Is noted that condition A and condition B are tight bounds that assume that the buffer must never be allowed 
1$ to overflow. Those skilled in the art recognize that looser constraints may be derived given that images are ol finite- 
length, rather than the continuoue stream assumed in the analysis. More relaxed constraints would permit more time 
for transcoding andfor allow less aggressive compression. 

[0089] It is thus an aspect of the present invention to provide a method lor a transcoding proxy to facilitate browsing 
between client devices and servers connected via a communication network. The method includes receiving an HTTP 

20 GET request from a client devices for an object stored at one of the servers, forwarding the GET request for the object 
to the server, receiving the object from the servers, examining preferences specified by a user of the client device, 
examining contents of the object, examining communication network characteristics, choosing a set of transcoding 
parameters, forming a transcoded form of the object, sending the transcoded form to the client, and/or examining 
network characteristics Including estimating network bandwidth between the server and the proxy as well as between 

2S the proxy and the client, and/or estimating delay between the servers and the proxy as well as delay between the proxy 
and the client device, and/or providing a feedback to the user about a level of transcoding performed on the object, 
and/or the step of examining including determining the size of the object, and/or determining dimensions of the object, 
and/or calculating the compresaion ratio of the object. If the object is of type image, dimensions of the image object 
are determined by area of the image in square pixels, and the compression ratio is determined by the bpp ratio of the 

90 image object. The present invention allows both store-and-forward and streaming transcoding, thus allowing forming 
a transcoded form before the step of receiving the object from the servers is complete. This method can be applied to 
JPEG, GIF and other image types. Another aspect of this invention is that it allows sending out the transcoded lorm 
before the step ol forming a transcoded form of the object Is complete. 

[0090] There are several other considerations that are important. The above examples for the concepts of the present 
as invention are usual lor image and video, etc. The wide use of the Internet has shown the value of JPEG and MPEG 
compreassd Image data. Audio coded data also needs to be decompressed, mixed with special sound effects, merged 
with other audio data, edited and processed in the real domain. Similar implementations are performed for other in- 
dustrial, commercial, and military applications 

[0091] This invention may also be provided as a process, an article of manufacture, apparatus, system, architecture 
*o and/or a computer product. For example, it may be implemented as an article of manufacture comprising a computer 
usable medium having computer readable program code means embodied therein for causing a computer to perform 
the methods of the present invention. 

[0092] It is noted that although the description of the invention is made for particular arrangements of steps, the 
intent and concept of the present invention are suitable and applicable to other arrangements. For example : the in- 
45 ventlon Is also adaptable to any browser although embodiment Is directed towards web browsing only. Although primary 
consideration is given to dynamic implementations, the invention maybe employed with a combination of static, quasi- 
dynamic and dynamic implementations. 



so Claims 

1 . A method for a transcoding proxy to facilitate browsing between a plurality of client devices and a plurality of servers 
connected via a communication network, the method comprising: 

55 receiving a request from one of the client devices for an object stored at one of the servers, 

forwarding the request for the object to said one of the servers, 
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receiving tho object from said one of the servers, 

examining preferences specified by a user of said one of the client devices, 
* examining contents of the object, 

examining communication network characteristics, 
choosing a set of transcoding parameters, 

10 

forming a transcoded form ol the object and 
sending the transcoded form to said one of the clients. 

2. A method as recited in claim 1, wherein the network characteristics includes bandwidth, and examining network 
characteristics includes estimating network bandwidth between said one ol the servers and the proxy ae well as 
between the proxy and said one of the clients. 

3. A method as recited in claim 1 or claim 2, wherein tho network characteristics includes delay, and examining 
so network characteristics includes estimating delay between said one of the servers and the proxy as well as delay 

between the proxy and said one of the clients. 

4. A method as recited in any preceding claim, further comprising providing feedback to the user about a level of 
transcoding performed on the object to form the transcoded form. 

25 

6. A method as recited In any preceding claim, wherein the step of forming a transcoded form employs dynamic 
adaptation. 

6. A method as recited in any preceding claim, wherein the step of forming a transcoded form is started before the 
so step ol receiving the object Irom said one of the servers is complete. 

7. A method as recited in any preceding claim, wherein the step of sending out the transcoded form is started after 
processing an Initial fraction of the received object, and before the step of receiving the Image object from said 
one of the servers is complete. 

35 . 

8. A method as recited In any preceding claim 1 , wherein the step ol sending out the transcoded form Isstartsd before 
the step of forming a transcoded form of the object is complete. 

9. A transcoding proxy system for facilitating browsing between a plurality of clients and a plurality of servers con- 
*o nected via a communication network, the proxy comprising: 

an HTTP proxy engine to receive a request from one of the clients for an object stored at one of the eervers : 
and to letch the object from said one of the servers, 

*s an object transcoder to form a transcoded form of the object using a set of parameters for transcoding, 

a dynamic policy module to determine the set of parameters of transcoding, 

an image size and delay predictor module to gather characteristics of the object, 

so 

a user preference module to gather quality preferences specified by a user of said one of the clients, and 

a bandwidth estimation module to estimate available network bandwidth, 

ss wherein said dynamic policy module dynamically adjusts the parameters of transcoding using the input re- 

ceived from the image size and delay predictor module, user preferences module, and bandwidth estimation 
module for the purpose of Improving satisfaction for the user, and the transcoding system provldee feedback 
to the user about the level of transcoding performed. 



13 

Received from < 1 408 490 2853 > at 10/18/03 12:50:19 AM [Eastern Daylight Time] 



From: John A. Smart 1 408 490 2853 To: TC 2100 - BF 



Data: 10/17/2003 Tlma: 8:35:48 PM 



Pag* 49 of 150 



EP0 992 922 A2 

10. A computer program comprising computor program instructions to cause a computer to perlorm the steps of the 
method as claimed In any ot claims 1 to 8. 
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